Path Integral Approach to Random Neural Networks

In this work we study of the dynamics of large size random neural networks. Different methods have been developed to analyse their behavior, most of them rely on heuristic methods based on Gaussian assumptions regarding the fluctuations in the limit of infinite sizes. These approaches, however, do not justify the underlying assumptions systematically. Furthermore, they are incapable of deriving in general the stability of the derived mean field equations, and they are not amenable to analysis of finite size corrections. Here we present a systematic method based on Path Integrals which overcomes these limitations. We apply the method to a large non-linear rate based neural network with random asymmetric connectivity matrix. We derive the Dynamic Mean Field (DMF) equations for the system, and derive the Lyapunov exponent of the system. Although the main results are well known, here for the first time, we calculate the spectrum of fluctuations around the mean field equations from which we derive the general stability conditions for the DMF states. The methods presented here, can be applied to neural networks with more complex dynamics and architectures. In addition, the theory can be used to compute systematic finite size corrections to the mean field equations.Comment: 20 pages, 5 figure

Analytical Solution of the Off-Equilibrium Dynamics of a Long Range Spin-Glass Model

We study the non-equilibrium relaxation of the spherical spin-glass model with p-spin interactions in the $N \rightarrow \infty$ limit. We analytically solve the asymptotics of the magnetization and the correlation and response functions for long but finite times. Even in the thermodynamic limit the system exhibits `weak' (as well as `true') ergodicity breaking and aging effects. We determine a functional Parisi-like order parameter $P_d(q)$ which plays a similar role for the dynamics to that played by the usual function for the statics.Comment: 8 pages, Roma preprin

Subextensive singularity in the 2D ±J\pm J Ising spin glass

The statistics of low energy states of the 2D Ising spin glass with +1 and -1 bonds are studied for $L \times L$ square lattices with $L \le 48$, and $p$ = 0.5, where $p$ is the fraction of negative bonds, using periodic and/or antiperiodic boundary conditions. The behavior of the density of states near the ground state energy is analyzed as a function of $L$, in order to obtain the low temperature behavior of the model. For large finite $L$ there is a range of $T$ in which the heat capacity is proportional to $T^{5.33 \pm 0.12}$. The range of $T$ in which this behavior occurs scales slowly to $T = 0$ as $L$ increases. Similar results are found for $p$ = 0.25. Our results indicate that this model probably obeys the ordinary hyperscaling relation $d \nu = 2 - \alpha$, even though $T_c = 0$. The existence of the subextensive behavior is attributed to long-range correlations between zero-energy domain walls, and evidence of such correlations is presented.Comment: 13 pages, 7 figures; final version, to appear in J. Stat. Phy

Response of an Excitatory-Inhibitory Neural Network to External Stimulation: An Application to Image Segmentation

Neural network models comprising elements which have exclusively excitatory or inhibitory synapses are capable of a wide range of dynamic behavior, including chaos. In this paper, a simple excitatory-inhibitory neural pair, which forms the building block of larger networks, is subjected to external stimulation. The response shows transition between various types of dynamics, depending upon the magnitude of the stimulus. Coupling such pairs over a local neighborhood in a two-dimensional plane, the resultant network can achieve a satisfactory segmentation of an image into ``object'' and ``background''. Results for synthetic and and ``real-life'' images are given.Comment: 8 pages, latex, 5 figure

Mutual Information of Population Codes and Distance Measures in Probability Space

We studied the mutual information between a stimulus and a large system consisting of stochastic, statistically independent elements that respond to a stimulus. The Mutual Information (MI) of the system saturates exponentially with system size. A theory of the rate of saturation of the MI is developed. We show that this rate is controlled by a distance function between the response probabilities induced by different stimuli. This function, which we term the {\it Confusion Distance} between two probabilities, is related to the Renyi $\alpha$-Information.Comment: 11 pages, 3 figures, accepted to PR

A Logic of Blockchain Updates

Blockchains are distributed data structures that are used to achieve consensus in systems for cryptocurrencies (like Bitcoin) or smart contracts (like Ethereum). Although blockchains gained a lot of popularity recently, there is no logic-based model for blockchains available. We introduce BCL, a dynamic logic to reason about blockchain updates, and show that BCL is sound and complete with respect to a simple blockchain model

On the conditions for the existence of Perfect Learning and power law in learning from stochastic examples by Ising perceptrons

In a previous letter, we studied learning from stochastic examples by perceptrons with Ising weights in the framework of statistical mechanics. Under the one-step replica symmetry breaking ansatz, the behaviours of learning curves were classified according to some local property of the rules by which examples were drawn. Further, the conditions for the existence of the Perfect Learning together with other behaviors of the learning curves were given. In this paper, we give the detailed derivation about these results and further argument about the Perfect Learning together with extensive numerical calculations.Comment: 28 pages, 43 figures. Submitted to J. Phys.

Statistical Mechanics of Support Vector Networks

Using methods of Statistical Physics, we investigate the generalization performance of support vector machines (SVMs), which have been recently introduced as a general alternative to neural networks. For nonlinear classification rules, the generalization error saturates on a plateau, when the number of examples is too small to properly estimate the coefficients of the nonlinear part. When trained on simple rules, we find that SVMs overfit only weakly. The performance of SVMs is strongly enhanced, when the distribution of the inputs has a gap in feature space.Comment: REVTeX, 4 pages, 2 figures, accepted by Phys. Rev. Lett (typos corrected

Large time off-equilibrium dynamics of a manifold in a random potential

We study the out of equilibrium dynamics of an elastic manifold in a random potential using mean-field theory. We find two asymptotic time regimes: (i) stationary dynamics, (ii) slow aging dynamics with violation of equilibrium theorems. We obtain an analytical solution valid for all large times with universal scalings of two-time quantities with space. A non-analytic scaling function crosses over to ultrametricity when the correlations become long-range. We propose procedures to test numerically or experimentally the extent to which this scenario holds for a given system.Comment: 12 page

Time Dependent Local Field Distribution and Metastable States in the SK-Spin-Glass

Different sets of metastable states can be reached in glassy systems below some transition temperature depending on initial conditions and details of the dynamics. This is investigated for the Sherrington-Kirkpatrick spin glass model with long ranged interactions. In particular, the time dependent local field distribution and energy are calculated for zero temperature. This is done for a system quenched to zero temperature, slow cooling or simulated annealing, a greedy algorithm and repeated tapping. Results are obtained from Monte-Carlo simulations and a Master-Fokker-Planck approach. A comparison with replica symmetry broken theory, evaluated in high orders, shows that the energies obtained via dynamics are higher than the ground state energy of replica theory. Tapping and simulated annealing yield on the other hand results which are very close to the ground state energy. The local field distribution tends to zero for small fields. This is in contrast to the Edwards flat measure hypothesis. The distribution of energies obtained for different tapping strengths does again not follow the canonical form proposed by Edwards.Comment: Minor changes and journal reference added. 10 pages 6 figure